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ABSTRACT 

The Division B vice-president of the American 
Educational Research Association discusses organizational changes in 
the division. Division H concentrates on evaluation and development. 
Recommendations deal with governance, publications, credibility, 
credent ia ling, ethics, leadership, and membership. (HLF) 
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What Do The Chickens Say Today ? 

The ancients found frequent need to consult with experts— hence the 
appearance in history of wise men, oracles and mystics* Roman emperors 
found experts who aided them in decision-making by reading the entrails of 
freshly slaughtered chickens. 

The modems have not lost the urge to seek consultant advice, and the 
science of such consultants may not be much advanced from the chicken 
stage* Practioneers and teachers of educational research, evaluation and 
related arts and skills have had some success in convincing decision- 
makers to use their knowledge and skill. However, it remains that signifi- 
cant decisions are made even without the benefit of chickens* 

Division H was formed with a nucleus of people who recognized this to 
be a problem. Many of the organizers of the Division were school-based 
R&D practioneers; others were university-based persons with a school- 
oriented philosophy— that the schools are where the action is; that changes, 
to be effective, must be where the action is; and that an effective way to 
promote desirable change in school practices is to make available R&D skills 
in the school community* 
^ Many school-based personnel scoff at the products of the university- 

^ based research community. The university seers produce answers to questions 
that haven *t been asked, or, when asked to provide answers to questions of Im- 

o 

portance, find it difficult to give straight forward, unequivocal responses 
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which are intelligible* Rather, the responses are often cloaked in the 
garb of a mystic cult, robed in statistical incantations* 

As out-going vice-president, and fully recognizing the difficulties 
entailed in doing things 1 am going to recommend, I will outline a sug- 
gested platform for Division H, acting within the umbrella of AERA, to 
attempt over the next several years. 

On governance — Hanv efforts have been made to study and change the 
governance structure of AERA, Some changes have been effected. Others, 
needed, have not. Some members have chosen to work through a known, exist- 
ing structure, and make it work toward their particular ends. Others have 
opted out, finding the structure unwieldy and unresponsive. I have found 
the organization to be more responsive than I thought it could be, although 
slow and reactionary at times. The notion that founding a division to deal 
with our interests captured our imagination and energy, and became a reality. 
One of the few benefits accruing from this action is the guaranteed space 
on the annual meeting program, and representation on the executive council 
of AERA. It does not guarantee that Division H members will find more 
journal articles meeting their needs, nor does it guarantee that AERA as an 
organization will meet their needs. It is interesting that over 1/4 of the 
membership of AERA select Division H as one of their Divisions of interest. 
This, however, merely serves to confuse the understanding of what and who 
Division H represents. Instead of a clearly identifiable community of in- 
terest, whose names and affiliation reveal their interest, we now have a 
vast array— or disarray— of diverse interests. Many seek to reform us and 
make us into proper researchers. 

As you know, all AERA members vote on all officer candidates. Thus, 
the vice-president of your Division, although selected by a nominating 
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committee appointed by the previous vice-president, is elected by vote of 
all AERA members. I do not believe this is yet a problem, but I believe 
it can become one, and would suggest that your new vice-president take a 
resolution to the next executive council meeting, recommending that divi- 
sion officers be voted for only by those electing membership in the divi- 
sion. I would further recommend that each divisional membership require 
the payment of an additional $1 above the basic AERA. dues, payable to AERA, 
and provided to the divisions as additional funds to conduct division busi- 
ness. The nominal amount would not prevent many from joining, but may 
separate idle curiosity and random form checking from a deliberate choice 
to affiliate. Perhaps a more identifiable constituency will result. Al- 
though recent council action established a pro-rata method for providing 
funds for a division budget, these funds are not adequate for other than 
minimal activities. 

Divisional vice-presidents and secretaries are elected. Each division 
has a program chairman, who is a major force in determining the extent of 
representation on the program, and in selection of papers and programs to 
be offered. The divisions also each have a student representative. Annu- 
ally, a nominatir. :oninittee appointed by the vice-president selects two 
candidates for whichever office is expiring. No other appointments are 
called for by the organization. However, the vice-president and the divi- 
sion can create whatever committees, task forces or positions they wish. 
I urge the division and its officers in the years ahead to use these means 
to accomplish several tasks, which I will outline. 

On publications— AERA official publications do not generally reflect 
the existence of Division H or its members in the content of articles. 
There are many reasons. Some are that Division H type articles and 
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interests do not meet the criteria of the journals, the editors, nor those 
of the reviewers. There is a need for an evaluation and development type 
journal, serving school-based or school-oriented practioneers. Such a 
journal could be established by the division, with sponsorship by an out- 
side agency, with or without official sanction of AERA. New publications 
are being encouraged by the AERA publications commit tee, and by the execu- 
tive council. Professor Jim Popham made such a proposal in his paper 
(yesterday) entitled "Career Development for Educational Evaluators: A 
Case of Advanced Undernourishment." I strongly urge the formation of an 
evaluator-developer journal by the division. 

On credibility— Many, too numerous to mention without running the risk 
of omitting an important contributor, have joined the argument over what 
is a credible evaluation ? and ^lo are credible evaluators? The several 
sides of the issue all have valuable points to consider. One such posi- 
tion is that premature closure may exclude methods or people which are 
more credible than what we've yet seen. Another position is that evalua- 
tion is subject to charlatans and quacks. One reason for both of these is 
the often unique and proprietary nature of evaluation studies and reports. 
The research conmunity depends on peer review to screen out or at least 
identify poor work. Evaluations are often not subject to peer review. 
Practioneers do not have a ready-made system for checking out their work, 
or for reading good work done by others, which could serve as a model. 
Often we are stuck with some idealistic statements of what a report should 
contain, or how a procedure should be carried out, with no way of knowing 
whether weVe hit the mark. Our judge is most often the consumer, whose 
concerns may be more public relations or administratively- oriented. 
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I recommend the establishment, in addition to the previously mentioned 
journal, of a division senior review board, to be composed of recognized 
evaluation talents, to which division member evaluators can submit manu- 
scripts of evaluation reports for critiques. This could be a monstrous 
workload and be totally unmanageable, but it could be a useful service con- 
tributed by senior members to the training and professional growth of junior 
members. The review board could select, on an annual b^sis, the top ten 
outstanding pieces of work, and the authors of those chosen then become 
members of the review board. Recognition would be given the authors at 
the annual division meeting. 

All who submitted manuscripts should receive some feedback to aid in 
their growth and development. Selections could be made for publication in 
the journal, and input made to ERIC under a special category— "Reviewed 
evaluation reports," including the critique. 

On credentialinfi— Credentigling evaluators and developers is a major 
problem. Some argue for credentialing by prescribed course work. Others 
take the position that credentialing should be done by competency tests. 
Still others opt for credentialing by product, and yet another school is 
the peer review process. I have no doubt that even more alternatives exist 
which are more or less viable. The division must foster the study of the 
credentialing of specialized personnel in evaluation and development, and 
must generate a feasible approach. Otherwise, the vacuum that exists will 
be filled by an agency over which we have no control. 

On ethics— Related to credentialing is the ethics problem. AERA has 
had an ethics conmittee. Its work and recommendations have been, and will 
continue to be made to the executive council. Its concerns are with the 
professional ethics of all AERA members. We must be concerned with a 



ERIC 

hiiiinniiinrrTiaaaa 



6 

narrower perr pective-the purview of Division H. I urge the establishment 
of a Division H ethics conrcittee, the chairman of which would be recom- 
mended to sit on the AERA ethics committee to provide for liaison. 

On leadership and membership; AERA and each division within it sur- 
vive by membership. At the same time, increasing numbers of members can 
increase diversity, and further complicate the mission of the division. 
However, there are undoubtedly hundreds of persons engaged in the practice 
of evaluation and development in public schools who are not now members of 
AERA or Division H. These persons, as members of AERA and the division, 
would provide a grass roots strength for the division, and support for such 
actions as creating a journal. I urge you to establish recruiting methods 
on a systematic, geographic basis which will encompass such practioneers. 

This will also provide a base for the development of my next recom- 
mendation, which is to establish a state-by-state organizational affiliate 
of Division H. Such state organizations need not be governed by AERA or 
the division. Such organizations would, however, draw support from local 
school distiTicts and universities. They would work at the state level to 
encourage good legislation and discourage bad legislation related to evalu- 
ation and development. As many have indicated, state legislatures have 
enacted accountability type laws, some without input or sanction from those 
whose task it is to implement such programs. Presently, revenue sharing 
funds for education have not been forthcoming. However, it is very likely 
that the states will become the focal point for financing education or for 
allocating funds from the federal government for education. Most education 
related organizations do have state level organizations. 

Few states, if any, have organized evaluator-developer organizations. 
Now is the time to move into this area, especially in the more heavily popu- 
lated states. 
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My final challenge is to local school districts, state education 
agencies and the federal governn:ent. Far too often, services such as those 
performed by evaluators and developers are only allowed when they appear 
to be required to get other funds or when it appears to be paid for by 
someone else. In times of short-fall funding, such services may appear to 
be unnecessary, a luxury, and the service downgraded or terminated. This 
at precisely the time when good information on the productivity of funds 
being spent is more ijnportant than ever, and decisions are to be made on 
reallocating funds, or making program cutbacks. Do not cut back on evalu- 
ation and development budgets. Do weed out incompetents and replace with 
the brightest, most creative people available. Do not put an unsuccessful 
teacher or administrator in such a position simply to have a place to put 
them. Do train your own or recruit well- trained, and/ or ejiperienced per- 
sonnel. 

It has been a professional privilege and social pleasure to have been 
your vice-president. I look forward to seeing the work of your new vice- 
president, Dr. Larry Barber, and your team. Thank you and good luck. 
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